Other articles

1. Animated Heart in Python

   read more

2. Expectation Maximisation in Python: Coin Toss Example

   Expectation Maximisation with Python : Coin Toss¶

   This notebook implements the example, I consider a classic for understanding Expectation Maximisation.

   See: http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html

   Notations:

   \begin{align*} \theta_A &= \text{Probability of a Heads showing up given the coin tossed is A}\\ \theta_B &= \text{Probability of a Heads showing up given the coin tossed is B}\\ \end{align*}

   read more

3. Entropy and Uniform Distribution

   To show \(\sum p_i\log(p_i) = p\)

   Consider \(H = -\sum p_i log(p_i) = log(n)\)

   Consider weighted AM-GM for \(p_i\):

   $$ (\prod \frac{1}{p_i}^p_i)^\frac{1}{\sum p_i} \leq \sum \frac{1}{p_i} \times p_i = n $$

   Take log both sides:

   $$ \sum p_i \log(\frac{1}{p_i}) \leq \log(n) $$ ...

   read more

4. A frequent inequality

   $$ \frac{x-1}{x} \leq \ln(x) \leq x-1 \forall\ x>0 $$

   Consider \(f(x)=\ln(x)-\frac{x-1}{x}\)

   \(f'(x) = \frac{1}{x} - \frac{1}{x^2} = \frac{x-1}{x^2}\)

   Now consider the following two cases:

   Case A: \(0 < x \leq 1\) and Case B: \(1 < x < \infty\) ...

   read more

5. Functions of independent random variables are independent

   A textbook problem

   Given \(X\), \(Y\) are two independent random variables, show that functions \(g(X)\) and \(h(Y)\) are independent too

   Short Proof

   Never took a course on measure theory, so avoiding that route:

   Let.

   \(R = g(X)\)

   \(S = h(Y)\)

   Also define,

   \(A_r = \{x: g(X)\leq r\}\) ...

   read more

6. Bernoulli random varlable and UMVUE

   Problem

   Let \(X_1,...,X_n\) random sample \(X\)~\(Bernoulli(p)\). For \(n\geq 4\) show that the product \(X_1X_2X_3X_4\) is a unbiased estimator for \(p^4\), and use this fact for find the best unbiased estimator of \(p^4\)

   Solution

   As posted on stackexchange

   [ToDo: Add variance, prove \(E[\phi(T)]=0\) ...

   read more

7. Distribution of one random variable less than other

   Problem

   \(P(Y<X)\) for any two independent random variables \(X,Y\)

   Solution

   We just follow the definition:

   \(P(Y<X) = \int_{-\infty}^{\infty}f_X(x)dx \int_{-\infty}^{x}f_Y(y)dy\)

   \(P(Y <X) = \int F_X(y)f_X(x)dx\)

   read more

8. [Proof]Weighted AM-GM

   We make use of Jensen’s Inequality and the fact that \(log(x)\) is a concave function:

   For concave f: \(f(\frac{\sum a_ix_i}{\sum a_i}) \geq \frac{\sum a_i f(x_i)}{\sum a_i}\)

   \(f=log(X)\)

   \(log(\frac{\sum a_ix_i}{\sum a_i}) \geq \frac{\sum a_i log(x_i)}{\sum a_i}\) ...

   read more

9. Expected number of pairings | Jones and Smith Family

   Problem

   4 members of Smith family and 4 of Jones family are pooled to form all possible pairs, all equally likely. Find \(N\), the number of smiths which have a smith partner.

   Solution

   \(X= \sum_{i=1}^4 I_i\)

   Where \(I_i=1\) iff \(i^{th}\) meber of smith family has ...

   read more

10. Poisson Demysitified

    Learnt it the hard way. This one is more of a scratch pad, inspired from the previously discussed problme on estimating the size of restriction fragments.

    Notations:

    \(N_t\)= Number of events that have happened in the \(t^{th}\) time interval

    \(T\) = Time taken till the first arrival occurs(or the ...

    read more

11. Prove: No UMVUE exists for $\frac{1}{\theta}$ for Poisson Distribution.

    Problem

    Proof: No UMVUE exists for \(\frac{1}{\theta}\) for Poisson Distribution.

    Solution

    http://stats.stackexchange.com/a/152660/11668

    Observation 1: \(\sum X_i\) is complete and sufficient statistic for \(\theta\)

    Observation 2: \(\sum X_i \sim Poisson(n\theta)\)

    We need to look for an unbiased estimator of \(\frac{1}{\theta}\) ...

    read more

12. Restriction Fragments, MLE, and Mixed Random Variables

    Let \(X_1, X_2, \dots , X_n\) be the lengths of \(n\) restriction fragments. Suppose that a biotechnique can measure fragment lengths accurately up to a given length c. That is, if \(X_i < c\), then the technique gives correct value \(X_i\) Show that MLE of \(\lambda = \frac{n-T_n}{S_n+cT_n}\) where \(S_n= \sum_{j=1}^n X_jI(X_j < c)\) ...

    read more

13. Restriction Fragments Size Distribution

    Given

    The number of restriction sites in a fragment of length \(t\) are distributed as:

    $$ P(N_t=k)=e^{-\lambda t}\frac{(\lambda t)^k}{k!} $$

    Find the distribution of length of restriction fragments

    Solution

    Intuition: If I have a restriction site at some location \(z\) and want it to ...

    read more

14. Convolution Demysitifed

    Problem

    Given \(f(x) = \frac{x}{2}\) for \(0 \leq x \leq 2\). Find the pdf of \(x_1+x_2\) for \(x_1,x_2\) which are i.i.d.

    Wrong solution

    $$ f*g(t) = \int_{-\infty}^{\infty} f(w)g(t-w)dw $$

    Thus, blindly,

    \(f_S(s) = \int_{0}^s \frac{x(s-x)}{4}dx = \frac{s^3}{24}\) ...

    read more

15. MATH 501 Project

    Introduction

    Model of a two state reparable system:

    $$ \begin{equation} \frac{dp_0}{dt} = -\lambda_0 p_0(t) + \int_0^1 \mu_1(x)p_1(x,t)dx + \int_0^1u^{*}(x,t)dx \end{equation} $$

    $$ \begin{equation} \frac{\partial p_1(x,t)}{\partial t} + \frac{\partial p_1(x,t)}{\partial x} = -\mu_1(x)p_1(x,t) - u^{*}(x,t) \end{equation} $$ ...

    read more

16. Runs in flips of a coin

    This problem happened to be in one of the screening examinations and is my favorite because it demonstrates an application of indicator variables

    Problem

    A run is defined as maximal subsequence of consecutive tosses all having the same outcome. So HHHTHHTTH has 5 runs.(HHH,T,HH,TT,H). Let ...

    read more

17. SVD v/s MDS v/s PCA

    Principle Component Analysis(PCA) is a relatively more famous than Singular Value Decomposition(SVD) or Multidimensional Scaling(MDS). When I was introduced to the latter two, I was utterly confused trying to figure out what goes in where.

    SVD

    Let \(X_{mxn}\) data matrix. For an easy to relate example ...

    read more

18. Multidimensional Scaling

    MDS is a statistical technique to visualize dissimilarity between points. The distances between two pointsin n-dimensions are visualized in 2 dimensions such that it represents the distance in n-dimensions as far as possible.

    It is important to note that, for MDS, we start of with a ‘distance’ matrix and not ...

    read more

19. Gumbel distribution expectation

    What

    If \(X_1,..,X_n\) is a random sample with density \(f(x;\theta)=e^{-(x-\theta)}e^{-e^{-(x-\theta)}}\) (\(x \in\mathbb{R}\)) and \(-\infty<\theta<\infty\), \(\quad\)i) Find the estimator of \(\theta\)

    Solution

    First let’s confirm if \(f(x;\theta)=e^{-(x-\theta)}e^{-e^{-(x-\theta)}}\) ...

    read more

20. BA = AB = I

    To Prove: If

    $$A_{nxn}$$

    and

    $$B_{nxn}$$

    such that AB=I, then BA=I

    $$AB=I \implies Rank(A), Rank(B)=n$$

    Reason: Rank(AB)

    $$\leq$$

    min(Rank A, Rank B)

    so B is a full rank matrix. Now consider B=BI

    $$\implies$$

    B-B(AB)=0

    $$\implies$$

    B-(BA ...

    read more

21. L2 norm and spectral radius of symmetric matrix

    L2 norm and spectral radius for

    $$A=A^T$$

    L2 inducded norm:

    NOTE: L2 norm is not the same as Frobenius norm. L2 norm is an “induced vector” norm, Frobenius is a “matrix” norm.

    Induced norm:

    $$$|| . ||$$

    defined as ||A|| =

    $$sup_{||x=1||} ||Ax||$$

    Note how ||Ax|| is a vector norm ...

    read more

22. Positive definite second derivative

    Let \(f:\mathbb{R}\to\mathbb{R}\) be a twice-differentiable function, and let \(f\)’s second derivative be continuous. Let \(f\) be convex with the following definition of convexity: for any \(a<b \in \mathbb{R}\):

    $$f\left(\frac{a+b}{2}\right) \leq \frac{f(a)+f(b)}{2}$$ ...

    read more

23. Proof for triangle inequality for case $x+y<0$

    \(-|x| \leq x \leq |x|\) and \(-|y| \leq y \leq |y|\) \(\implies\) \(-|x|-|y| \leq x+y \leq |x|+|y|\) \(\implies\) \(|x+y| \leq |x| +|y|\) for any real \(x,y\)

    The last implication comes from the fact: \(|x| \leq a \leftrightarrow -a \leq x \leq a\) for some \(a \geq 0\) ...

    read more